Audio-Visual Perception in Interactive Virtual Environments

Interactive virtual environments (VEs) are gaining more and more fidelity. Their high quality stimuli undoubtedly increase the feeling of presence and immersion as “being in the world”, but maybe they also affect user’s performance on specific tasks. Vision and spatial hearing are the main contributors of our perception. Sight dominates clearly and has been in the focus of research for a long time, but maybe it is the audiovisual combination which facilitates the user in his decision making and in completing a task. Mere identification of the task is not enough. Of course one could find dozens of problems where spatial sound reproduction has a practical relevance. More interesting are those which reside on a high cognitive level. Tasks that combine visual stimuli and auditive perception with movement provide a wide field of activity like for example crossing a busy road, an every day task that contains a high information density and demands fast processing by the brain. But how does hearing have an impact on this? Does spatial audio lead to better performance? Can one adjust naturalistic, spatialized hearing virtually? This diploma thesis asseses the effect of spatial sound reproduction compared to conventional stereo sound or no sound at all. Within the scope of the practical part, a simulator was implemented to produce a virtual street crossing experiment. It was later used to perform a study with volunteer participants. The results give evidence that there is a statistically significant difference between spatialized sound rendering compared to stereo sound or no sound. In the future this can not be used solely to boost the naturalistic fidelity and authenticity of a virtual environment but also as a user supportive measure.